Search for: All records

Algorithmic bias research often evaluates models in terms of traditional demographic categories (e.g., U.S. Census), but these categories may not capture nuanced, context-dependent identities relevant to learning. This study evaluates four affect detectors (boredom, confusion, engaged concentration, and frustration) developed for an adaptive math learning system. Metrics for algorithmic fairness (AUC, weighted F1, MADD) show subgroup differences across several categories that emerged from a free-response social identity survey (Twenty Statements Test; TST), including both those that mirror demographic categories (i.e., race and gender) as well as novel categories (i.e., Learner Identity, Interpersonal Style, and Sense of Competence). For demographic categories, the confusion detector performs better for boys than for girls and underperforms for West African students. Among novel categories, biases are found related to learner identity (boredom, engaged concentration, and confusion) and interpersonal style (confusion), but not for sense of competence. Results highlight the importance of using contextually grounded social identities to evaluate bias. 

Students' reading ability affects their outcomes in learning software even outside of reading education, such as in math education, which can result in unexpected and inequitable outcomes. We analyze an adaptive learning software using Bayesian Knowledge Tracing (BKT) to understand how the fairness of the software is impacted when reading ability is not modeled. We tested BKT model fairness by comparing two years of data from 8,549 students who were classified as either "emerging" or "non-emerging" readers (i.e., a measure of reading ability). We found that while BKT was unbiased on average in terms of equal predictive accuracy across groups, specific skills within the adaptive learning software exhibited bias related to reading level. Additionally, there were differences between the first-answer mastery rates of the emerging and non-emerging readers (M=.687 and M=.776, difference CI=[0.075, 0.095]), indicating that emerging reader status is predictive of mastery. Our findings demonstrate significant group differences in BKT models regarding reading ability, exhibiting that it is important to consider—and perhaps even model—reading as a separate skill that differentially influences students' outcomes."]} 

Abstract BackgroundSelf-regulated learning (SRL) strategies can be domain specific. However, it remains unclear whether this specificity extends to different subtopics within a single subject domain. In this study, we collected data from 210 college students engaged in a computer-based learning environment to examine the heterogeneous manifestations of learning behaviors across four distinct subtopics in introductory statistics. Further, we explore how the time spent engaging in metacognitive strategies correlated with learning gain in those subtopics. ResultsBy employing two different analytical approaches that combine data-driven learning analytics (i.e., sequential pattern mining in this case), and theory-informed methods (i.e., coherence analysis), we discovered significant variability in the frequency of learning patterns that are potentially associated with SRL-relevant strategies across four subtopics. In a subtopic related to calculations, engagement in coherent quizzes (i.e., a type of metacognitive strategy) was found to be significantly less related to learning gains compared to other subtopics. Additionally, we found that students with different levels of prior knowledge and learning gains demonstrated varying degrees of engagement in learning patterns in an SRL context. ConclusionThe findings imply that the use—and the effectiveness—of learning patterns that are potentially associated with SRL-relevant strategies varies not only across contexts and domains, but even across different subtopics within a single subject. This underscores the importance of personalized, context-aware SRL training interventions in computer-based learning environments, which could significantly enhance learning outcomes by addressing the heterogeneous relationships between SRL activities and outcomes. Further, we suggest theoretical implications of subtopic-specific heterogeneity within the context of various SRL models. Understanding SRL heterogeneity enhances these theories, offering more nuanced insights into learners’ metacognitive strategies across different subtopics. 

Adaptive learning systems are increasingly common in U.S. classrooms, but it is not yet clear whether their positive impacts are realized equally across all students. This study explores whether nuanced identity categories from open-ended self-reported data are associated with outcomes in an adaptive learning system for secondary mathematics. As a measure of impact of these social identity data, we correlate student responses for 3 categories: race and ethnicity, gender, and learning identity—a category combining student status and orientation toward learning—and total lessons completed in an adaptive learning system over one academic year. Results show the value of emergent and novel identity categories when measuring student outcomes, as learning identity was positively correlated with mathematics outcomes across two statistical tests.